Advances in Pass Rush and Tackling Evaluation in American Football with Player Tracking Data

.title[
# <small>Advances in Pass Rush and Tackling Evaluation in American Football with Player Tracking Data</small>
]
.subtitle[
## <small>Quang Nguyen</small>
]
.institute[
### Department of Statistics & Data Science <br> Carnegie Mellon University <br> <br> JSM 2024
]
.date[
### <svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M459.37 151.716c.325 4.548.325 9.097.325 13.645 0 138.72-105.583 298.558-298.558 298.558-59.452 0-114.68-17.219-161.137-47.106 8.447.974 16.568 1.299 25.34 1.299 49.055 0 94.213-16.568 130.274-44.832-46.132-.975-84.792-31.188-98.112-72.772 6.498.974 12.995 1.624 19.818 1.624 9.421 0 18.843-1.3 27.614-3.573-48.081-9.747-84.143-51.98-84.143-102.985v-1.299c13.969 7.797 30.214 12.67 47.431 13.319-28.264-18.843-46.781-51.005-46.781-87.391 0-19.492 5.197-37.36 14.294-52.954 51.655 63.675 129.3 105.258 216.365 109.807-1.624-7.797-2.599-15.918-2.599-24.04 0-57.828 46.782-104.934 104.934-104.934 30.213 0 57.502 12.67 76.67 33.137 23.715-4.548 46.456-13.32 66.599-25.34-7.798 24.366-24.366 44.833-46.132 57.827 21.117-2.273 41.584-8.122 60.426-16.243-14.292 20.791-32.161 39.308-52.628 54.253z"/></svg> <span class="citation">@qntkhvn</span>       <svg aria-hidden="true" role="img" viewBox="0 0 496 512" style="height:1em;width:0.97em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M165.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6zm-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3zm44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9zM244.8 8C106.1 8 0 113.3 0 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C428.2 457.8 496 362.9 496 252 496 113.3 383.5 8 244.8 8zM97.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1zm-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7zm32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1zm-11.4-14.7c-1.6 1-1.6 3.6 0 5.9 1.6 2.3 4.3 3.3 5.6 2.3 1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2z"/></svg> qntkhvn       <svg aria-hidden="true" role="img" viewBox="0 0 640 512" style="height:1em;width:1.25em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M579.8 267.7c56.5-56.5 56.5-148 0-204.5c-50-50-128.8-56.5-186.3-15.4l-1.6 1.1c-14.4 10.3-17.7 30.3-7.4 44.6s30.3 17.7 44.6 7.4l1.6-1.1c32.1-22.9 76-19.3 103.8 8.6c31.5 31.5 31.5 82.5 0 114L422.3 334.8c-31.5 31.5-82.5 31.5-114 0c-27.9-27.9-31.5-71.8-8.6-103.8l1.1-1.6c10.3-14.4 6.9-34.4-7.4-44.6s-34.4-6.9-44.6 7.4l-1.1 1.6C206.5 251.2 213 330 263 380c56.5 56.5 148 56.5 204.5 0L579.8 267.7zM60.2 244.3c-56.5 56.5-56.5 148 0 204.5c50 50 128.8 56.5 186.3 15.4l1.6-1.1c14.4-10.3 17.7-30.3 7.4-44.6s-30.3-17.7-44.6-7.4l-1.6 1.1c-32.1 22.9-76 19.3-103.8-8.6C74 372 74 321 105.5 289.5L217.7 177.2c31.5-31.5 82.5-31.5 114 0c27.9 27.9 31.5 71.8 8.6 103.9l-1.1 1.6c-10.3 14.4-6.9 34.4 7.4 44.6s34.4 6.9 44.6-7.4l1.1-1.6C433.5 260.8 427 182 377 132c-56.5-56.5-148-56.5-204.5 0L60.2 244.3z"/></svg> qntkhvn.netlify.app
]

---

---

# The challenges and the opportunities

Defensive performance has been an understudied part of American football

Challenges&#58;

* Previous metrics: box-score statistics or based on subjective judgment

* Not all [INSERT STATS] are created equal

* Positions like defensive linemen is sufficiently lacking recorded statistics

Opportunities&#58; tracking data provided by the NFL Big Data Bowl

* 2023 theme: linemen on pass plays

* 2024 theme: tackling

---

<p style="font-size:4em; color: #C41230">
<b>
Pass rush evaluation
</b>
</p>

---

# Team

]

]

---

# Previous metrics for pass rush evaluation

*   Play-level stats: sacks, hits, and hurries

*   Pass rush win rate

*   uses an arbitrary time thresold to define a pass rush win

*   converts continuous data to a win-loss indicator

---

# football meets materials science

---

# football meets materials science

---

# football meets materials science

---

# measuring pressure at the frame-level

**Definition** `$\text{(STRAIN}$`; informal)

For every pass rusher at each frame within a play, calculate

*   `$d$`: distance between pass rusher and QB

*   `$v$`: velocity at which pass rusher is moving toward QB
    
--

*   `$\text{STRAIN} = \displaystyle- \frac{v}{d}$`

---

# Advantages of STRAIN

*   Simple, computation friendly: 2 features! #PutThatInYourXGBoost

*   Interpretable: 1/STRAIN <br> `$\rightarrow$` time required for a pass rusher to get to QB with current velocity & distance

*   Scalable: can be applied to every passing play (minus trick plays)

*   Continuous-time within-play metric

*   Properties: discrimination, predictability, face validity

---

# data: nfl big data bowl 2023

Player tracking data for the first 8 weeks of the 2021 regular season

Example play: Raiders vs Broncos (week 6) – T. Bridgewater sacked by M. Crosby

---

# example play for Raiders DE <span style="color: #1143E2">Maxx Crosby</span>

---

---

# STRAIN differentiates between positions (edge & interior)

---

# STRAIN differentiates between play outcomes

---

# STRAIN is more predictive of pressure than pressure itself

---

# STRAIN is highly stable over time

---

# Multilevel model for STRAIN

`$$\small \begin{aligned} \overline{\text{STRAIN}}_{ij} &\sim N(R_{j[i]} + B_{b[ij]} + D_{d[i]} + O_{o[i]} + \mathbf{x_{ij}} \boldsymbol{\beta}, \sigma^2); \text{ } i = 1, \dots, n \text{ plays} \\ R_{j} &\sim N(\mu_R, \sigma^2_R); \text{ } j = 1, ..., \text{# of rushers} \\ B_{b} &\sim N(\mu_B, \sigma^2_B); \text{ } b = 1, ..., \text{# of blockers} \\ D_{d} &\sim N(\mu_D, \sigma^2_D); \text{ } d = 1, ..., \text{# of defenses} \\ O_{o} &\sim N(\mu_O, \sigma^2_O); \text{ } o = 1, ..., \text{# of offenses} \end{aligned}$$`

- Response: average STRAIN for pass rusher at the play-level

- Random effects: pass rusher, pass blocker, defensive team, offensive team

- Fixed effects: number of blockers involved in a play, position of blocker and rusher, play-context (down, yards to go, current yardline)

---

# Pass rusher rankings obtained from resampling

---

<p style="font-size:4em; color: #C41230">
<b>
Tackling evaluation
</b>
</p>

---

# Team

---

# EXISTING TACKLING METRICS ARE FLAWED

* Tackles (solo or assisted) are unofficial stats

* StatsBomb treats assisted tackles as 0.5

* PFF created "stops"

* PFF tracks "missed tackles"

These are only discrete counting stats!

---

# What is the purpose of tackling?

---

# Tackling evaluation framework

* Purpose of tackling: to halt the **forward motion** of the ball carrier

* Measuring forward motion: **velocity** toward the end zone

Proposal&#58; 3-step model-free framework, aiming to

*   Measure tackling contribution throughout a play

*   Assign defensive credit for halting ball carrier’s forward motion

*   Provide a continuous metric for defensive performance

Data&#58; First 9 weeks of the 2022 regular season; only consider RB run plays

---

# example play for Giants RB Saquon Barkley

---

# STEP 1: identifying contact windows

---

# STEP 1: identifying contact windows

---

---

# Step 2: valuing a contact window

---

# Step 2: valuing a contact window

---

# Step 3: crediting individual players

---

# Fractional tackles summary for example play

---

# TACKLES and ASSISTS ARE OVERSTATED STATISTICS

---

# Fractional tackles possess great stability

---

# Does momentum (size) matter?

---

<p style="font-size:4em; color: #C41230">
<b>
Challenges still exist
</b>
</p>

---

# Lesson learned: let's take a step back

> The more experience I&#39;ve gained with player-tracking data, the more I realize the importance of simplicity. This is not statistical work - but rather focusing on the foundation: observed data.<br><br>Next steps: statistical work!...<br><br>&mdash; Ron Yurko (@Stat_Ron) <a href="https://twitter.com/Stat_Ron/status/1744483411505414583?ref_src=twsrc%5Etfw">January 8, 2024</a>

<!--

tracking data: rich and complex

forget the model, what are the observables? what are we actually observing

more work should be spent on this

how are teams going to handle as we collect more and more data

what are the "sufficient statistics" here

-->

---

# We still got work to do!

Look at what baseball has done...

*   Statcast tracks exit velocity and launch angle

<!--

smoothed measurements from technology

-->

*   Now Statcast bat tracking is publicly available

*   Data available via Baseball Savant

Can football (Next Gen Stats) do the same thing?

This remains a challenge...

---

# It's really that simple

---

# Related links

*   Papers

*   STRAIN: [arxiv.org/pdf/2305.10262](https://arxiv.org/pdf/2305.10262)
    
    *   Fractional tackles: [arxiv.org/pdf/2403.14769](https://arxiv.org/pdf/2403.14769)
    
*   Big Data Bowl submissions: [STRAIN](https://www.kaggle.com/code/statsinthewild/strain-sacks-tackles-rushing-aggression-index) & [Fractional tackles](https://www.kaggle.com/code/tindata/momentum-based-fractional-tackles)

<p style="font-size:1.2em; color: #C41230"> <b> Carnegie Mellon Sports Analytics Conference (#CMSAC) </b></p>

*   November 1—2, 2024

*   Register at [stat.cmu.edu/cmsac/conference](https://www.stat.cmu.edu/cmsac/conference)

---

<p style="font-size:4em; color: #C41230">
<b>
Appendix
</b>
</p>

---

# Valuing a contact window

The value for a contact window is defined as `$\displaystyle \frac{v_{\text{start}} - v_{\text{end}}}{v_{\text{pre}}} \,.$`

*   If `$v_{\text{pre}}$` happens within window: replace `$v_{\text{start}}$` with `$v_{\text{pre}}$`

*   If `$v_{\text{post}} \ge v_{\text{pre}}$`: window has zero value

*   If `$v_{\text{end}} \le v_{\text{post}} < v_{\text{pre}}$`: replace `$v_{\text{end}}$` with `$v_{\text{post}}$` (fraction of unrecovered peak velocity)